Double-Dummy Analysis Problems


This document discusses problems with using Double-Dummy Analysis for computer bidding. For more background information, see the BidBase Background file.

1. It can take a very long time for the program to come up with and analyze enough sample hands to get a meaningful estimate of which bid is best. This makes playing against the program slow and boring. The alternative is to limit the amount of time it may use to "think", which speeds up the bidding, but seriously degrades the quality of the bidding because the software doesn't have enough time to generate a statistically valid number of hands which match the criteria of the bids already made.

In the May 2002 issue of Bridge World, page 47, for the bidding P-2-P-P, Dbl-P-?? and the hand QJ2 AQ743 K73 63, the magazine says that its DDA program: had a significant amount of trouble generating a sample. This translates to a very long wait for a bridge-playing game to bid this hand using DDA, not to mention that without a statistically valid sample, the results are meaningless.

2. Using randomly generated hands may not result in the same bid each time a similar hand (or even the exact same hand) comes up due to statistical anomalies in DDA, so the bidding sequence may not be repeatable. (I have personally run across this problem in trying to point out bidding errors to the creator of GIB, which relies most heavily on DDA.)

3. The longer the bidding goes, the less accurate (and more time-consuming) DDA becomes. This is because in each random deal, the program must re-perform DDA on earlier bids in the bidding sequence which it did before to get to the current point in the bidding. The inherent inaccuracy in computing each bid is cumulative, resulting in less and less accuracy. Also, as noted in #2, above, the DDA may not get the same bid as for an earlier deal, causing the best bid to be rejected, even though the deals are virtually identical.

4. DDA does not generate bids, only hand/contract evaluations. At best, DDA can tell you what the optimum contract might be (and not even that, in every case), but it does not tell you the best bid to make to get there.

Programmers historically have used hard-coded algorithms to decide what to bid based on the DDA results.

Programs which use hard-coded algorithms, even with the use of DDA, are limited in the extent to which they can support different bidding systems and conventions. If you like to open 1NT with (14)15-17 HCP (defined as normally 15-17, but occasionally 14), but the program only supports 15-17 point NT openings, there is nothing you can do about it.

In fact, at any point in the bidding, no matter how good the DDA is, if the programmer specifies a bid to make which you don't like, or is simply a gross error, there is nothing you can do about it.

5. DDA used with hard-coded algorithms cannot remember past analysis, bids, or results. So if it takes several minutes to make a bid in a particular situation, it will take the same length of time every time it faces the same situation.

Image 6. DDA doesn't play exactly like humans would. For example, if there is a 2-way finesse, DDA will always take it the correct way. Another example, on defense, DDA can, for example, see whether or not a ruff will be forthcoming by leading a singleton, or whether it is safe to cash A-K in a suit when either partner or opener may be the one to ruff the 3rd round, etc., etc.  
To the right is an example of how unrealistic DDA can be. The very large percentage of the time, N-S will get 6 clubs, 2 diamonds, and 1 heart for a laydown 3N. If east leads a spade, 4N is laydown.  
Yet the DDA (at the bottom) says that north can only make 8. 
Look it over and see if you can spot how to hold N-S to 8 tricks.
Click here for the answer.


Image 7. First, the DDA for the specific lay of the cards to the left shows that N-S can make slam, but when we get an average DDA by mixing E-W's cards 10 times and recomputing the DDA each time, we see that slam will not make on average. 
Also, there are 2-way finesses in both hearts and spades. A human player can only expect to make the finesses the correct way half the time, but the computer will always take them the correct way, even when the E-W cards are remixed, so it would be more accurate to say that the optimum should be one less than shown.. 
Second, everyone will bid Stayman -
2N-3, 3-3N, Pass.
However, the DDA shows N-S's optimum is game in the 4-3 heart fit, but there is no way to bid it playing normal Stayman. 
Playing against a program which uses DDA for bidding, it may bid 4 over 3NT in this situation. It is bad to have a program which makes abnormal bids (or plays) in order to score a top because it teaches bad habits.


Image 8. DDA for bidding is not accurate if it doesn't construct hands matching the other player's bidding.  
In the deal to the right, south opens 3H with a 7-card heart suit. DDA for just the lay of the cards as shown says that east can only make 2N, but the 11-interation averaging and even the 101-iteration averaging says that east can make 3N. 
The reason is that when the N-S cards are mixed for averaging, south's 7-card heart suit is not maintained. If the DDA took south's bid into consideration and left the heart suit in south's hand, it is unlikely that 3NT could ever make. It's not even clear how 5 could make more than half the time, as shown in the DDA averaging columns. E-W must lose a heart, and 2 diamonds. even when the spade finesse works. 
But the problem with trying to take the bidding into consideration is that it requires the software to know what each bid means, but bids can have a virtually unlimited number of meanings thanks to the availability of different conventions and even the variety of different possible bids in "standard" bidding. (Look at It's Your Call in the ACBL Bridge Bulletin).



From The Horse's Mouth...

Long after the above was written, we received a brochure advertising a bridge program by the name of Bridge Buff. Ironically, the brochure highlighted almost all of the above problems in its own program:

Bridge Buff might find another bid if it bids the same hand a different time. On some auctions, it might decide to bid, or pass, or double depending on the sample of simulated deals. The auctions are much more cluttered, the opponents more active, and some bids might seem unusual or aggressive, but most [emphasis ours] are not so goofy that you wouldn't see them in a typical matchpoint game. The bidding will not be instantaneous.

The disclosure that most bids are not so goofy means that some of them are so goofy. We appreciate the brochure's candor, but it seems a little strange to actually be advertising the quirkiness of DDA.

The Bridge Buff web site says:

    Bridge programs usually use a set of rules for bidding, or a large database of bids... For a given hand, the program will always make the same bid because it steps through the same lines of code. Holding say Axx-KQJxx-Qxxx-x, it will always overcall 1 because that rule says to. The program will never change its mind

    One would think that if there is one best call for a certain situation we would want the program to always use it rather than taking a long time to calculate a possibly different call which, by definition,would not be the best call.

    And if there is not one best call, it is not true that a non-DDA program must always use the same bid.

    As can be seen below, in BidBase, the same hand has entries for 3 different calls based on the voting by experts in the It's Your Call column in the Feb. 2019, ACBL Bridge Bulletin. The "Pct" (percent) column reflects the voting for the different calls. When BidBase sees this, it will select one of those entries the specified percentage of the time. For example, 36% of the time it will pass.

    Bridge Buff goes on to explain how it uses simulation and DDA to see how, in the situation above, the partnership fares in a heart contract, compared to the opponent's playing in their contract. Here, the program will probably decide that it gains more by bidding and playing 1 than by letting the opps play in 1, so overcaller will bid 1... The bidding will not be instant, because Buff 'thinks' about the bids.

    The flaw in this is that rarely is opener bidding 1 in hopes of playing a club contract. In fact, very often, opener will bid 1 with only 3 or 4 clubs. So all this "thinking" to come up with 1 to keep opener from playing 1 seems like a waste of time.

    This is not to say that there is not a place for simulation and DDA. In fact, as we have said elsewhere, BidBase is designed to allow the entry and customization of any and all conventions which normally only requires specifications for a couple of rounds of bidding, and that beyond that, something like simulation and DDA is needed, though it also appears possible that a different type of bidding database based on hand data revealed by previous bidding might work.